How To Monitor & Track Google’s Real-time Search

Questions I’d love to see the answer to are things like:

• What triggers real-time one-boxes?
• How long do they last?
• Are they tied to a geographic location?
• Are they tied to a language?

Unfortuantely I think it’s a bit early to have answers to questions like this, so rather than tackle these questions I’m just going to talk a little bit about how you can go about tracking the impacts of real-time search results on your industry.

Does real-time search affect my industry?

The answer is probably yes. For search terms that have hardly any tweet-volume I’ve already seen examples where literally one or two tweets can generate a real-time one-box. Sometimes even for the brand name term. That means that more or less any breaking news in your industry will generate some level of real-time results.

But what about other industries? After all many of us will be working on sites that target keyphrases that people DO tweet about. For us, the focus is on trending search terms. The key thing is to identify the types of keyphrase that might feature real-time search results. The most useful way of doing this that I’ve found is to monitor twitter volume and in particular monitor peaks and troughs in volume. Trendistic will do this nicely for you. The first neat thing from Trendistic is that you can see a long list of hot topics by day in the archive:

Browsing through the archives we see that there are certain topics which come up again and again such as TV, film, sports, celebrity etc. These search terms are alwasy going to be affected most by real-time search and SEOs working in this field are likely to already be used to working with QDF search results and various other one-boxes like News.

How Do I Track Real-Time Traffic?

The second nice thing from Trendistic is the ability to query individual terms and see when peaks and troughs occured over time, for example here’s a snapshot of the [eagles] term (nice win Eagles!):
By using a service like this you can query the historic search volume and take an educated guess at when real-time search might have been triggered. By doing this for your main search terms you can start to understand things like strange traffic drops or spikes that might have been caused by real-time one boxes hanging out in your SERPs.

What about if you’re actively engaging in twitter though? If you feel like you might have gained a portion of your search traffic from tweets that were appearing in real-time search results then you should think about tracking those clicks.

Tracking real-time search volume and one-box traffic is a difficult problem however and one that isn’t completely solved. That said, here’s a few things that might be of use. Firstly, for anyone seeing #-based Google URLs you can actually track clicks from different parts of the page. Looking at the following real-time search for [nexus one]:

I clicked on two different results, the first one was a ‘real’ result that appeared in the real-time box, that is a page that’s been crawled recently and shows up via Google rather than showing up because Google found the result on Facebook or Twitter etc. With the # URLs at Google in action I saw the following full referral path:

http://www.google.co.uk/url?sa=t&source=web&oi=blog_result&ct=result&cd=11&ved=0CBcQmAEwCg&url=http%3A%2F%2Fwww.ccortez.com%2Fhtc-nexus-one-blessed-by-the-fcc-updated%2F&rct=j&q=nexus+one&ei=gComS7LCDZehjAeDwdTOBw&usg=AFQjCNF2939x_yuKVTzL9UlN6m23cw0Kog

Note the “&oi=blog_result” in the referring URL (bolded added, obviously). This let’s you see any real-time traffic that has come via a crawled blog post. After that I clicked on a twittered URL and got the following:

http://www.google.co.uk/url?url=http://bit.ly/7315xj&rct=j&ei=2yomS4y7NYvNjAfQ3qXfBw&sa=X&oi=microblog_result&resnum=9&ct=result&cd=1&ved=0CD8QoAQoADAI&q=nexus+one&usg=AFQjCNGWb9DkQaPZd2NGuOg6Th7lWd9hsg

Note both the url=http://bit.ly/7315xj and &oi=microblog_result (again, bolded). This allows you to see both where the click came from (a real-time microblog result, i.e. from a site like twitter or facebook) but also the URL that was twittered (in this case the bit.ly link).

These referring URLs will show up in your server logs but unfortunately won’t show up in Google Analytics (since Google treats these all as search queries and so will just dump them in the same place and only let you see the keyword searched for). To get them to show up in Google Analytics you need to set up a profile to show the full referring URL, such as the filter detailed in part 2 of this post.

Not all users see these # Google URLs however, most are still seeing the old style search?q= Google URLs. From looking at the traffic for sites where we have the appropriate filter set up I’d say somewhere between 5 and 10% of users are seeing these URLs. This means that if you can get this kind of data for a small proportion of your traffic and extrapolate for the other 90% of users. (Btw, does anyone have any more accurate stats on the % of users seeing which search result type? I’ve not seen anything concrete anywhere…)

Of course, looking at the example above we see that a fair amount of traffic from micro blogging servicies actually goes through URL shorteners such as bit.ly. In that case there’s another method you can use to track your traffic. Take a look at the following referral list for this bit.ly URL:

This allows you to see which of your bit.ly links have appeared on Google search results pages – we can see from this example that 2 have come from new # style Google search results pages and one has come from the old-school format.

I’m sure over the coming weeks more and more will get said about real-time search but hopefully this has been food for thought!

If you haven’t yet grabbed your copy of our new Advanced SEO Training Series: Tips, Tricks & Tactics DVD series, there’s good news! SEOmoz extended the special launch pricing of 20% off plus free shipping until December 18th. Order your copy now before the offer is gone!